Generating Random Graphs with Large Girth 



Mohsen Bayati* Andrea Montanari^ Amin Saberi^ 



Abstract 

We present a simple and efficient algorithm for randomly 
generating simple graphs without small cycles. These 
graphs can be used to design high performance Low-Density 
Parity-Check (LDPC) codes. For any constant k, a < 
l/2k{k + 3) and m = 0{n^^"), our algorithm generates an 
asymptotically uniform random graph with n vertices, m 
edges, and girth larger than k in polynomial time. To the 
best of our knowledge this is the first polynomial-algorithm 
for the problem. 

Our algorithm generates a graph by sequentially adding 
m edges to an empty graph with n vertices. Recently, 
these types of sequential methods for counting a nd random 
generation have been very successful |35l Ha [nl [3 m E] ■ 

1 Introduction 

We present efficient algorithms for generating random 
simple graphs with cycles of size larger than a constant 
k. The main motivation for this work comes from the 
design of high performance Low-Density Parity-Check 
(LDPC) codes [31.. 

For positive integers m, n, k, our algorithm gener- 
ates a random graph with n vertices and m edges that 
has no cycles with length less than or equal to k us- 
ing 0{n^m) operations. The algorithm starts with an 
empty graph and sequentially adds m edges between 
pairs of non-adjacent vertices. In every step of the pro- 
cedure, an edge can be added between two distinct ver- 
tices i and j that are of distance at least k. The prob- 
ability of adding an edge between i and j, denoted by 
Pij , changes in every step of the algorithm. In order to 
get a uniform sampling, pij should be proportional to 
the number of extensions of the current graph to graphs 
with m edges that contain (ij) and have no small cy- 
cles. The algorithm estimates the number of such valid 
extensions by computing the expected number of small 
cycles if the rest of edges are added uniformly at random 
(see Sections [3] and |4] for more details) . 

We show that our algorithm produces an asymp- 
totically uniform sample when fc is a constant and 
m = 0(n^+") with a < l/2fc(fc + 3). To the best of our 
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knowledge this is the first polynomial-time algorithm 
for the problem. The analysis uses Janson's inequal- 
ity for obtaining a close approximation of the number 
valid extensions of a partially constructed graph. This 
method goes beyond the Poisson approximations that 
is typically used when the number of the edges is linear 
(see |2] for more details). 

From a theoretical perspective, our work is related 
to the following problem. Given a graph property P 
that is preserved by removal of any edge from the 
graph. A random maximal P- graph is obtained from 
n isolated vertices by randomly adding those edges 
(at each stage choosing uniformly among edges whose 
inclusion would not destroy property P) until no further 
edges can be added. The question of finding the number 
of edges of a random maximal P-graph for several 
properties P is well studied [33l [Ml IMl HI El]- In 
particular, when P is the property that the graph has 
girth greater than fc, [27 shows that the above process 
of sequentially growing the graph leads to graphs with 
m = 0(n^+'^ logn) edges. 

Unfortunately, these random maximal P-graphs 
may have distribution that are far from uniform. In 
fact it has been shown (e.g. {36j) that when P is the 
property of having no triangle, the maximal triangle 
free graphs are close to bipartite. We show that 
our new algorithm guarantees asymptotically uniform 
distribution at the expense of reducing the number of 
edges to m = 0{n^^^^<^). 

Recently, sequential algorithms have been shown, 
empirically ([III [11) and theoretically f[35l \T8\ \E[ 15]). 
to be very successful for designing fast algorithms for 
counting and generating random graphs with given de- 
grees. The current paper builds on this line of research 
and develops mainly two new ideas. For the design, in- 
stead of starting from a biased heuristic and calculating 
and canceling its bias we use Poisson approximations to 
obtain the correct pij's. For the analysis, we use convex 
functions and Janson's inequality for controlling the ac- 
cumulated error and obtaining a tighter bound than the 
concentration results used in [S] [TB] . 

1.1 Application in designing LDPC Codes It 

has been shown that LDPC codes can approach Shan- 



non capacity asymptotically for large size codes, when 
their associated graph representations (Tanner graphs) 
are selected uniformly at random from the set of bipar- 
tite graphs with a properly optimized degree sequences 
[121 121| . However, in practice, the maximum graph size 
is between 10'^ and 10^ (depending on the delay sensi- 
tivity and on the hardware constraints). In this range, 
it is well known that the existence of a small number of 
subgraphs with a certain structure (in particular, small 
cycles) spoil the code performances 1301 123 ■ 

Different approaches have been developed within 
the coding theory community to deal with this prob- 
lem. For example, deterministic constructions of graph 
sequences with large girth [261 122] have been studied. 
However, numerical studies have shown that known de- 
terministic constructions can have poor performance 
[22] . From a theoretical point of view, no deterministic 
graph sequence is known that asymptotically outper- 
forms random graphs. 

One can also stick to random constructions and 
grow the graph by adding random edges sequentially 
while avoiding short cycles. This method has been 
very popular in practice and is known by the name of 
progressive edge growth (PEG) algorithm |14j . We will 
describe the main intuition behind PEG and show its 
limitations with respect to two standard performance 
measures for the codes: (i) bit error rate or expected 
fraction of wrong bits; and (ii) block error rate or 
probability that at least one bit in the message was 
received incorrectly. 

Let Citcr be the maximum rate achievable by ran- 
dom LDPC codes (empirically Cucr is indistinguish- 
able from the channel capacity). It is known that uni- 
formly random graphs contain a random number (of 
order 0„(1)) of cycles of size k or smaller. These cycles 
are responsible for non- vanishing block error probability 
that is bounded away from at small noise. The main 
goal of PEG is to reduce this error, to a value that van- 
ishes with by removing the cycles of length up to k. 
But the final distribution of PEG is not necessarily uni- 
form which may affect the other performance measure 
(bit error rate) . In fact preliminary simulations suggest 
that our new algorithm produces codes with lower bit 
error rate. 

In this paper we define the first code generation 
algorithm that overcomes both problems. We show 
that there exists a graph sequence that (1) can be 
generated efficiently; (2) has vanishing bit error rate 
at any rate below Citcr (this follows by the standard 
density evolution analysis [3T] using optimized degree 
sequences [TH HI]); and (3) has girth larger than k 
(therefore has low block error rate probability). 

For the sake of simplicity we will present the rele- 



vant calculations only for the problem of generating ran- 
dom (not necessarily bipartite) graphs that have large 
girth. Then for LDPC design, we will define the algo- 
rithm for generating bipartite graphs with given degree 
sequences. Generalizing proofs to this case is cumber- 
some but is conceptually straightforward. 

1.2 Organization of the Paper. The rest of the 
paper is organized as follows. In Section [51 we provide 
the setup and necessary definitions. In Section [31 we 
describe the new algorithm for randomly generating 
large-girth graphs, and state the main result. In Section 
[4] we explain the intuition behind the algorithm, and 
present its application to LDPC codes in Section [5] 
Finally the analysis of the algorithm and its running 
time are discussed in Section [6] To the interest of space 
some of the proofs are presented in the longer version 
of the paper [4j. 

2 Definitions and Problem Statement 

The girth of a graph G is defined to be the length 
of its shortest cycle. Let G„_m denote the set of all 
simple graphs with m edges over n vertices and let 
Gn,m,fe C 'Gn,m bc the sct of graphs with girth greater 
than k. In this paper we assume that fc is a constant. 
We want to sample a uniformly random graph G from 
Gn,m.fe- This is a relatively easy problem for m — 0{n) 
and in this paper our focus is on the difficult case of 
m = 0(n^+") for some small a > 0. 

A nai've approach would be to start with an empty 
graph Go with no edges over the vertex set V = 
{l,2,...,n} and sequentially add pairs (ij) chosen 
uniformly at random (without replacement) among all 
pairs {ij) that are not yet selected. Repeating the 
edge addition m times leads to a uniformly random 
member of Gn,m that has girth larger than k, with 
probability bounded away from 0. Re- running the whole 
m steps a sufficiently large number of times, yields 
a polynomial-time algorithm for uniformly randomly 
generating graphs in Grn,m,k when m — 0{n). This 
approach does not work for the case of m = 0(n^+") 
since the success probability is of the order e~" . For he 
applications to coding theory, even the case m = 0{n) 
is very challenging in practice since n is very large. 

3 Algorithm and Main Result 

Our sequential algorithm to sample from Gn.m,k works 
as follows. 

Algorithm S 



(1) Set Go to be the graph over vertex set V ~ 
{1, 2, . . . , n} and with no edges. For t = 0, . . . ,m — 



1, repeat the following steps: 

— If there is no edge (if) such GtU{ij) £ 'Gn^t+i,k 
(adding any edge to Gt creates a small cycle) , 
stop and return FAIL. 

— Otherwise, consider the probability distribu- 
tion p{ij\Gt) given by equation (|3.1[) below 
on the set of all edges (ij). Sample an edge 
(ij) with distribution p(ij|G() and set Gt+i = 
GtU{ij). 

(2) If the algorithm does not FAIL before t = m — 1, 
return Gm- 

Let Mt and AI^ be the adjacency matrix of the 
partially constructed graph Gt and its complement G^ 
respectively. Let also St be the adjacency matrix of 
all edges (ij) such that Gt U (ij) £ Gn,t+i,k- Define a 
probability distribution on the set of all pairs {ij) and 
represent it by a symmetric matrix Pq^ — [p{ij\Gt)] 
defined by 



(3.1) 
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Here Z(Gt) is a normalization constant. Symbols and 
exp represent the coordinate-wise multiplication and 
exponentiation of square matrices, i.e. for n x n matrices 
B, C the expression A = B Q G {A = exp{B)) means 



that for all 1 < i,j < n we have ai 



The main intuition for defining the probabilities 
p(ij\Gt) through matrix multiplication is explained in 
Section m 

By construction Gm G (G„,„i,fc. Let Ps(G) denote 
the probability that algorithm S does not FAIL and 
returns a graph G G G„^m,fc- Let also P[/ denote the 
uniform probability on the set Gn,m,k', i-e. Vu{G) = 
\'Gn,m,k\~^- Our goal is to show that Ps(G) and Pc/(G) 
are very close to each other and for that we will use the 
total variation metric. 

Definition The total variation distance between two 
(not necessarily normalized) measures P and Q on a set 
5* is defined by: 

dTy(P,Q)=sup {|P(A)-Q(A)| : AcS}. 

Our main result is the following: 

Theorem 3.1. For m = and constant fc > 3 such 

that a < [2k{k + 3)]~^ , the failure probability of algo- 
rithm S is o(l). Moreover, the algorithm generates all 



but o{\Gr„^rn.k\) graphs in Gn.m,k with equal probability 
asymptotically. In particular we will show that: 



lim drvi^ 



= 0. 



We will prove Theorem 13.11 in Section [6l We will also 
discuss an implementation of algorithm S in Subsection 
6.21 that has an expected running time of 0{n^m). 



4 The Intuition Behind Algorithm S 

Define the execution tree T of the naive algorithm 
described in Section [2] as follows. Consider a rooted 
m-level tree where the root (the vertex in level zero) 
corresponds to the empty graph in the beginning of 
the algorithm and level r vertices correspond to all 
couples (GrjTTr) wherc Gr is a partial graph that can 
be constructed after r steps, and iTr is an ordering of its 
r edges. There is a link in T between a partial graph 
{Gr,T^r) from level r to a partial graph (Gr+i,T^r+i) 
from level r + 1 if G,- C Gr+i and tt^, tt^+i coincide 
on the first r positions. Any path from the root to a 
leaf at level m of T corresponds to one possible way of 
generating a random graph in Gn,m- 

Let us denote those partial graphs Gr that have 
girth greater than k by "valid" graphs. Our goal is to 
reach a valid leaf in T, uniformly at random, by starting 
from the root and going down the tree. It is known 
that [34] in order to achieve this goal, at any step r, one 
needs to choose Gr+i = G^ U {{ij)} with probability 
proportional to the number of valid leaves of T that 
are descendant of {Gr+i,Trr+i) (see [5] for a similar 
analysis in more details). Denote this probability by 

P(Gr+l, TTr+l). 

The main technical contribution of this paper is de- 
riving a new approximation p{Gr+i,'^r+i) for the true 
probabilities p(Gr+i, 7r,.+i), selecting (Gr-+i, tt^+i) with 
probability p(Gr+i, 7r,._).i), and controlling the accumu- 
lated error Y[7=o^ [p(Gr+i, 7r,.+i)/p(Gr+i, tt^+i)]. 

Consider a random variable nk{Gr+i,T^r+i) that is 
the number of cycles of length at most A: in a leaf 
chosen uniformly at random from the descendants of 
(Gr+i,7rr+i) in T. First we use Janson's inequality 
[2] to show that the distribution of nfc(Gr+i, tt^+i) 
behaves like Poisson. That is the probability of 
nk{Gr+i,'!Tr+i) — (i.e. reaching a valid leaf) is ap- 
proximately exp (— E[nfe(Gr+i, TTr+i)]). That explains 
how p(Gr+i, TTr+i) is Calculated. 

Furthermore, instead of calculating the error 
[p{Gr+i,'n'r+i)/p{Gr+i,TTr+i)] for cach r and then 
bounding the accumulated error, we look at the final 
product, simplify it, and then find a bound for it. For 
this problem, the standard method of bounding each 
term separately, leads to much larger error terms and 



is not sufRcient for deriving our final result. In order 
to make the implementation easier, it is not hard to 
see that the terms E[nfe(G'r+i, TTr+i)] can be approxi- 
mated by matrix multiplication (more precisely by en- 
tries of the matrix Ylll^-i {^^t + jSy^M^'^ used in equa- 
tion (|3.ip ). These claims will be rigorously proven in 
Section [6l We believe that this idea is applicable for 
the analysis of similar random generation algorithms 
beyond their existing correctness ranges. 

5 Application to LDPC codes 

The ideas described above can be used to generate 
random bipartite graphs with given degree sequences. 
Such graphs define the standard ensemble for irregular 
LDPC codes. Here we will show how to modify the 
algorithm for this application without repeating its 
cumbersome, but conceptually simple, analysis. 

Consider two sequences of positive integers f — 
ri , . . . , r„ and c = ci , . . . , Cm for degrees of the vertices 
such that e = J27=i'''i — SjLiCj- We would hke 
to generate a random bipartite graph G{Vi,V2) with 
degree sequence (f, c), (i.e. for Vi = {ui, . . . , V2 — 
{vi, . . . ,Vm} we need deg{ui) — and deg{vj) — Cj) 
that has also girth greater than k (assume k is an even 
number). We denote the set of all such graphs by Gf^c.k- 
The algorithm is a natural generalization of Algorithm 
S where the probabilities p(ij|G't) are adjusted to obtain 
the elements of set Gf^c,k- 



Algorithm Bip-S 



(1) Set Go to be a graph over vertex sets Vi — 
{ui, . . . ,Un}, V2 = {vi, . . . ,Vm} and with no edges. 
Let also f — {fi, . . . , f„} and c = {ci, . . . , Cm} be 
two ordered set of integers that arc initialized by 
f = f and c = c. For t = 0, . . . ,e — I, repeat the 
following steps: 

— If there is no suitable edges, i.e. For any edge 
{uiVj), the graph Gt U {uiVj) has a cycle of 
lenght at most k, stop and return FAIL. 

— Consider the probability distribution 
q{uiVj\Gt) given by equation (|5.2p below 
on the set of all edges {uiVj). Sample an edge 
[uiVj) with distribution q{uiVj\Gt) and set 
Gt+i = GtU {uiVj). 

(2) If the algorithm does not halt before t = e — 1, 
return Gg. 



Here each probability q{uiVj\Gt) is an approximation 
to the probability that a uniformly random extension of 



graph Gt has girth larger than k. In order to find this 
approximation, we will consider a configuration model 
representation for the graphs with degree sequence (f, c) 
(see [5] for the definition of configuration model). Then 
we use the argument of Section 21 to find the following 
Poisson approximation for q[uiVj\Gt)'- 

(5.2) qMGt)^ ''^ ^^^^^ 

where Z{Gt) is a normalization constant, and fi, Cj, de- 
note the remaining degrees of i and j. Furthermore, 
Ek{Gt,Ui,Vj) = E^=iE7eC2,-^{K''.)67}a(7,Gt,Ui'yj) 
where C2r is the set of all simple cycles of length 
2r in the complete bipartite graph on vertices of Vi 
and V2, and a^j, Gt,UiVj) is roughly the probabil- 
ity that 7 is in a random extension of Gt- More 
precisely a{'-f,Gt,UiVj) is a product of the three 
terms '"''^^jZt-^l')^'^^' ^ Uuie^y^i'^i^ "f^^t^u.Vj), and 
Ilvi,£-yH'"s,l,Gt,UiVj) where b{ui,j,Gt,UiVj) is equal 
to Ijfi, or fi{fi — l) depending on whether both, one, or 
none of the adjacent edges to ui in 7 are in Gt U {uiVj) 
respectively. b{vs,"f,Gt,UiVj) is defined similarly. 

We defer a more complete discussion of the codes 
generated by this algorithm to a complete version of 
the paper. Here we limit ourselves to a few remarks: 
(1) Several definitions have been proposed for the sub- 
structures responsible for the decoding errors at high 
signal-to-noise ratio. Our algorithm can be adapted to 
exclude any of these substructures (instead of cycles) as 
well. (2) In any of these definitions, the cycles play a 
dominant role. Therefore the above algorithm should 
be a good starting point. (3) In practical code design 
it can be preferable to partially structure the ensemble 
for facilitating the layout (as, for instance, in proto- 
graph codes |31j). Our graph generation procedure can 
be adapted to partially structured ensembles as well. 

6 Analysis 

The aim of this section is to prove Theorem 13.11 The 
most challenging part of the proof is to show that Pg (G) , 
probability of generating a graph G by Algorithm S, is 
not less than P[/(G), the uniform probability, for almost 
all graphs G in Grn,m,k- In fact, the two parts of the 
proof of Theorem 13.11 are given below. There is also a 
brief analysis of the running time of the algorithm which 
is discussed in Subsection 16.21 

Lower bound for Pg. The probability of gen- 
erating a typical graph in Gn,m,k is at least a con- 
stant fraction of the uniform probability. More pre- 
cisely we will show that for all but o{\Gn,m,k\) graphs 
G e <Gn,m,k- IPs(G) > (1 - o(l))Pc/(G) where the term 
0(1) goes to zero quickly as n goes to infinity. Proof 



of the lower bound covers almost all of this section and 
as we can see in the next paragraph the main theorem 
follows easily from it. 

Proof of dTv{^s,'Pu) = 0(1). Once the above 
lower bound is given to us it is not hard to show 
that Ps and Pg are close to each other in total vari- 
ation metric. This part is an straightforward alge- 
braic calculation. We just need to apply the trian- 
gle inequality to the definition of dTv{^S,^u) and ob- 
tain dTv(Ps, Pc/) < EgsG,, ,„ JPs(G) - Pc/(G)|. Then 
depending on whether Ps(G) > Pc/(G) or Ps(G) < 
(l-o(l))Pc/(G) we bound the term |Ps(G)-Pc/(G)| dif- 
ferently. Let D C G„ m fc be the set of graphs G where 
the Ps(G) < (1 - o(i)jPc/(G) and let B c G„,™.fe be 
the set of all graphs G with Ps(G) < P[/(G)! Now 
assuming the lower bound on P5 given above the fol- 
lowing facts hold: (i) \D\ — o{\Gnmk\)- (ii) For 
G e B\D : \Fs{G) - Pu{G)\ = Ps(G) - Pc/(G) < 
o(l)Pt/(G). 

(6.3) \^s{G)-¥u{G)\^ 

^ (Ps(G)-Py(G)) + 2^ |Ps(G)-PLr(G)l = 
(Ps(G)-Pc/(G)) + 2 Y |lPs(G) -Pc/(G)| 

GSG„_„,fc GeB\D 

+ 2 Y |Ps(G)-Pa(G)| 
GeBnD 

< J2 ^s(G)- J2 p^(^) 

+ o(l)^P,,(G)+2^ 1 

GeB Geo 

<l-Ps(FAIL)- 1 + 0(1) +0(1) 

< 0(1) -Ps(FAIL) < 0(1). 



Moreover, since the left hand side of the equation 
is non-negative then the probability of failure of 
the algorithm S , denoted by Ps(FAIL), is o(l). This 
finishes proof of Theorem 13.11 I 



6.1 Lower Bound For Ps(G) This proof contains 
five main steps and the most important ones are the 
Steps 4 and 5. Before entering the details we give 
a high level description of the analysis. Since the 
probabihties of selecting the edges (denoted by p{ij\Gt) 
) are calculated by a Poisson approximation, they are 



of the form 



Where each random variable 



Et{ij) or Et{rs) refers to the expected number of 
elements in a family of cycles (see Step 2 for the details) . 
We partition the union of these families of cycles for 
all steps of the algorithm into two subsets depending 
on whether they are from a numerator term Et{ij) 



or a denominator term Et{rs). We also consider a 
third set of cycles corresponding to non-suitable pairs 
(referred to by forbidden pairs in Step 3 below). The 
probability Ps(G) can be shown to be proportional to 
gSi+S2+S3 where each of ^i, S2 and ^3 is a function of 
the cycles in a unique family among the three families 
discussed above. We will show a combinatorial 1 to 1 
correspondence between the cycles appearing in Si, S2, 
and the ones appearing in S'3 with negative signs. This 
simplifies the summation 81 + 82 + 83 and produces the 
desired lower bound independent of the graph G. 

Step 1: Using Jensen's Inequality. The first 
step is to write an expression for the probability that 
the algorithm S does not fail and returns a fixed graph 
G G <Gn,m,fc- Note that algorithm S sequentially adds 
edges to an empty graph to produce a graph with 
m edges. Hence for the fixed graph G there are ml 
permutations of the edges of G that can be generated by 
algorithm S and each permutation can have a different 
probability. Let tt be any permutation of the edges of G 
(i.e. a one-to-one mapping from {1, . . . , m} to the edges 
of G), and let GJ be the graph having V as vertex set 
and {tt{1), . . . ,Tr{t)} as edge set. This is the partial 
graph that is generated after t steps of the algorithm S 
conditioned on having tt as output. Now we can write 
Ps(G) = E.UT=o' Pi^it + l)|Gr)- Assuming that tt 
is a uniformly random permutation then the term 
can be replaced by mlE^ where is expectation with 
respect to the uniformly random permutations. Thus 
we have 



(6.4) 



m — 1 



Ps(G) = m!E^ I Yl p{7:{t + l)\G^)j 
= m!E, exp \ogp{^{t + l)|Gr)| 

(m-l 

> ml exp I J2 E^log]5(7r(< + l)|Gn 



t=o 



where the inequality is by Jensen's inequality for the 
convex function e^. 

Step 2: Estimate for the probabilities p{Tr{t + 

1)|GJ^). Recall from Section[3J that in algorithm S, after 
t steps, the probability of adding an edge (ij) to a par- 
tially constructed graph Gt should be proportional to 
the number of uniformly random extensions of Gt U (ij) 
to a graph in G„,m.fc. Using the Poisson approxima- 
tion this number is roughly proportional to e^^'=*^'^''*-'^ 
where Ek{Gt,ij) is defined to be the expected number 



of simple cycled of length at most k if we add m — t—l 
random edges to Gt U (ij). Define N = (2) . It is easy to 

see that E,{Guij) = Y!t=:,T.Zl where 
= and A^^^''^'"'-* is the number of simple cycles in 
the complete graph on V that have length r, and include 
{ij) and exactly I other edges of Gt- 

In fact the rigorous statement is given by the 
following lemma and its proof is given in [4] . 

Lemma 6.1. For any non-zero probability termp{ij\Gt) 
we have p{ij\Gt) > ^e-^''(G-^^)-0("'*'^"°") 

where Z[Gt) — X^rs summation over 

all suitable pairs (rs) at step t. 

Note 1. In the definition oi p{ij\Gt) in Section [3] we 
used matrix multiplication to count number of cycles 
which counts non-simple cycles as well. This is because 
it makes the implementation of the algorithm easier. 
For the analysis it is convenient to work with simple 
cycles and the above lemma provides the comparison 
that we need between these two methods of counting 
cycles. 

Step 3: Algebraic modifications. Now we use 

the estimates given by Lemma 16.11 in equation (|6.4p to 
obtain: 



modified lower bound for VsiG): 



Ps(G) > m!exp 



(fc(fc+3) + l)Q-l^ 



log Z(Gr)-0(n 



By condition a < [{k(k + 3) + 1)]~^ the third term of 
the exponent is o(l). Now we are going to simplify the 
second term of the exponent. Let us define Zo(GJ^) — 
N -t- Y!t=s.Fr{G1) where Fr{G1) is number of all 
forbidden (unsuitable) pairs at step t. Now we drop the 
reference to G^ ) for simplicity and obtain: 



logZ = logZo+log(Z/Zo) 



log 



{N-t){l- 



N -t 



logiZ/Zo) 



(a) F 

< log(7V - t) - ^'•^■'^ 



log(Z/Zo) 



(b) V F 

< log(7V_t)_^£i^ + log(Z/Zo) 

where (a) uses log(l — x) < —x for x G [0, 1] and (6) 
uses N — t < N . Using the above we will the following 



^ Cycles that do not repeat a vertex or edge. 



(6.5) Ps(G)> 



■ exp 



^E,£;fe(Gr,7r(f+l)) 



m— 1 



1 — M +— n f — n *^ ^ i ' 



To simplify the notation, we will denote the three 
terms in the exponent by Si{G), S^iG), Sz{G) respec- 
tively. In particular: Si{G) = - E^o^ ¥.^Ek{G1 ,T:{t + 



1)), S2{G) = ^Et3Erro'E.^r(Gr) and 53(G) 

^m-lTj^ Z{G^ 



Next two steps are the most important parts of 
our effort in proving the inequality P5(G) > (1 — 
o(l))Pt/(G). 

Step 4: Simplifying the exponent iS'i(G) -I- 
<S'2(G)-|-iS'3(G). This step shows the main benefit of leav- 
ing the calculation of approximation errors for p{ij\G^) 
to final steps. We will show that even though the terms 
Si{G) for i = 1,2,3 can be large and dependent on 
graph G, their sum can be simplified to an expression 
which is independent of graph G. In fact we will show 
that 5*1 and S2 will be completely canceled by the graph 
dependent parts of S^. 

First we are going to find a lower bound for the SiS. 
These lower bound are given with the following lemma. 

Lemma 6.2. Let m — and constant k > 3 be 

such that a < [2k{k + 3)] ^ then for all but o{\Gn,m,k\) 
number of graphs G G Gn^m.k the followings hold: 



(a) Si(G) > 0(1) - 

EtaE^ri |c...(G)| £j^e'-\i~ey-'de. 

(b) S2{G) > 0(1) + Ets |c.,.-i(G)!f XI (^"^''de. 

(c) SaiG) > 0(1) + 

j:'^=sJ:Zo\CrAGmy-'{r^e)fle'{i-ey-'-'de. 

where Cr/{G) is set of all simple cycles of length r in 
complete graph on V that include exactly I edges of G. 

Let us start by providing a high level over view of the 
proof of this lemma and for now we just focus on S\(G). 
By definition 5i(G) = - E^o^ ^^^(G'r, 7r(t -f- 1)). The 
first approximation we use is to change the randomness 
given by tt. The partial graph G1 is a uniformly random 
subgraph of G that has exactly t edges. Instead we 
look at Ge which is a random subgraph of G that has 
each edge of G independently with probability 9 = t/m. 
The subgraph Gg has t edges in expectation which 
makes it a good approximation for G^. With this 
substitution, one can see that — E"=o ^i^EkiGI , Tr{t + 



1)) is roughly equal to ~mEg d9Ek{Ge, (ij)) where 
{ij) is a uniformly random edge of G. 

The next steps focus on expanding the Ek- Ek is 
summation of the terms qt~^~^ for couples 
where 7 is a simple cycle of length r on complete graph 
on V that has £ edges of Gg and (ij) is an edge of G\Gg 
that is in 7 as well. For any fixed r, £ it can be shown 
that the expected number of such couples is dominated 
by the cases where |7nGe I = l7nG| — 1 =£ that is when 
(ij) is the only edge ofGHj that is not in Gg. Moreover 
Qt is approximately equal to (1 — 6)m/N. Therefore the 
contribution of the pairs {"^jij) in Si{G) for any fixed 
r, i (i.e. for all 7 G Cr^e) is the same. Thus we obtain 
the right hand side of the Lemma [62Ia)- 

The above discussion is proven rigorously in [4J. 
Now we will show how Si + S2 + S3 is simplified using 
the above lower bounds. 

Step 5: Finishing the proof of Pg > (1 
o(l))Pc/. First we provide an estimate for number of 
graphs in the set Gn,m,k- Using Janson's inequality 
[2] one can obtain close approximation for number of 
graphs in the set Gn,m,k- In fact Janson's inequality 
shows the number of cycle of constant length in Grn,m 
converges to a Poisson random variable. The following 
Lemma formalizes this statement. 



Lemma 6.3 
such that a < {2k 



Let m = n^^°' 



I exp 



r- 



and constant k > Z he 
- 1)^^ ■ Then \'Gn.m,k\ is equal to 



Where Ct- is the set of all simple cycles of length r in 
the complete graph on vertices of V. Proof of Lemma 
6.31 is given in Appendix lAl 

Next we will show that how different terms in 
the lower bounds for Si^s in Lemma 16.21 cancel each 
other. The main key in relating the terms in those 
lower bounds of SiS is the following equation which is 
obtained using integration by parts for r — I > £ > I: 



(6.6) 



if 9^-^{i-eY-^de = {r-£) j e\i 

Jo Jo 



'd0 



Therefore, combining equation (|6.5p and Lemma 16.21 for 
all graphs G in Gn.7n,k that satisfy Lemma [6.21 fall but 
0(1) fraction of the elements in Gin,m,k) we obtain: 



Ps(G) > 



> 



,Si{G)+S2{G)+S3{G)+o{l) 

3°(i)+E'=3 \Cr,o{G)\{^Y rf^^i^ey-^dd 
71v\ 



(6.7) 



(W^eO{i)-E"=3lc..o(G)l(t)'' 



where the second inequality is the result of equation 
(|6.6p and some algebraic cancelations as follows. All 
terms in the lower bound for Si with 1 < £ < r — 2 
are canceled with the corresponding terms in the lower 
bound for S3, and the £ — r—1 term in the lower bound 
of 5*1 is canceled with the lower bound of ^2. Therefore 
the uncanceled terms in Si{G)+S2{G)+S3{G) are £ = 
terms from the lower bound of S3. 

Comparing the right hand side of the equation (|6.7p 
and the expression for |G„_m,fe| given by Lemma 16.31 
we see that the only difference is the use of Crfi{G) 
instead of C^. But note that for a cycle C € Cr the 
probability that it intersects graphs G is at most r 
times the probability that a fixed edge of C intersects G 
(union bound). Therefore P„,,„(C n G ^ 0) < rm/N = 
0{n"~^). This shows the following: 



N' 



>_0(,('=+i)-i) + ^,C,.|(^)'- 



= o(l)+ElC,.|(^) 



Thus, we obtain 



Ps(G) > 



where (h) uses Lemma [ 



> (l-o(l))Pc/(G) 



6.2 Running time of the algorithm. The fact that 
Algorithms S has polynomial running time is clear since 
the matrix of the probabilities, , at any step can 
be calculated using matrix multiplication. In fact a 
naive calculation shows that Pg^ can be calculated with 
0{kn^) = 0{n^) operations. This is because for any 
r takes 0{rn^) operations to compute. So we obtain 
the simple bound of 0{n^m) for the running time of the 
algorithm S. But one can improve this running time by 
at least a factor n with exploiting the structure of the 
matrices. 

Recall that the matrix Pg^ is proportional to St 



exp 

matrix Mt 



-Mi 



-M- by Xt. 



Let us denote the 
We also denote the 



adjacency matrix of the partially constructed graph Gt 
at step t by At. Notice that the adjacency matrix of 
all "suitable pairs" at step t (denoted by St) is equal to 
Jn — sign(^^^Q Al) where J„ is the n by n matrix of all 
ones and the operation sign(_B) for any matrix B means 
that the "sign" function is applied to each entry of the 



matrix B. This is correct since any non-suitable pair 
(ij) corresponds to a path in Gt of length r between i 
and j for < r < A: — 1. Such path forces the ij entry 
of the matrix to be positive. 

Now we can store the matrices At,...,Af^^ 
and Xf,...,Xf~^ at the end of each iteration and 
use them to efficiently calculate At^i, . . . , A'^^^ and 
Xf_^-^, . . . , X^^^ . This is because the differences At+i — 
At and Xt+i — Xt are very sparse matrices and hence 
updating the matrix multiplications can be done with 
0{tt?) operations which reduces the overall running time 
to 0{n^m). 

Since Theorem 13. f I shows that algorithm S is suc- 
cessful with probability 1 — o(l) then the expected run- 
ning time of the algorithm S for generating an element 
of the set Gn,m,k is 0{'n?m) as well. A similar idea can 
extended to the algorithm Bip-S for generating LDPC 
codes. We leave a detailed analysis of the running time 
and implementation of these algorithms to the longer 
version of the paper. 
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A Estimate For 



^n.m,k I 



and proof of Lemma 



First we approximate the random graph model Gn,m 
with the Erdos-Reyni model Gn,p for p = m/N. We will 
denote the probability with respect to the randomness 



in Gn,p and 



by 



and 



respectively. Let 



G(n,p) or G{m,n), has girth greater than k. First 
we will calculate the probability ¥n,p{Ak). Then we 
will formalize the following approximation to obtain 
estimates for \Gn,m,k\- ^n,p{Ak) « |G 



A.l Janson's Inequality. In this section we briefly 
review Janson inequality. Given a collection of 'bad 
events' {Bi : i G I}, we would like to estimate 
the probability P( fligi Bf) assuming that the events 
Bf, i G I are "almost independent". More formally, 
let (3, 7 be real numbers such that [3 < 1 and for all 
i el: P(B,) < P and Eb.^b, ^(^^ ^1 Bj) = 7 where 
Bi ^ Bj means that the events Bi,Bj are dependent. 
Then Janson's inequality is the following: 
(1.8) 

7 



nP(S- ) < P( n.ei B^) < Y[FiB^) exp 



2(1-/3) 



In particular, for 7 — o(l) we obtain: P( fligi i?,f) = 

(i+o(i))n,eiW)- 

A. 2 Finding P„ p(ylj.) via Janson Inequality. By 

definition, Pn,p{Ak) is the probability that a random 
graph G in Gn,p has no cycle of length at most k. Define 
the set of all cycles of length r in complete graph on V by 
Cr. Let C = uj5^3C,- and consider the set of bad events 
Bi, i G C where Bi is the event that G contains the cycle 
i. It is not difficult to see that: F{Bi) = 0{p^) and 

7 = O (X^r^a V*^"^) • And since p = 0{n"-^) 
then using Janson's inequality (jl.Sp we obtain: 



UnB^) < p„,,(A,) < e^i-''"-'"--') unB^) 



which gives the following for a < l/{2k — 1): 



fnAAk) = e 



o{l) 



n 



n(> 



lcngth{i) 



(1.9) 



exp 



exp 



o(l)+^|C.llog(l-p^ 

k 

o(l)-^|C,.l? 



Ak be the event that a random graph, selected from 



where the last equality uses jCrlp^*^ = 0{n/' n^''"^^'") = 
0(1) since a < 1/2. 

Finally we are ready to prove Lemma 16.31 which is 
achieved by approximating |Gn,m,fc| through Pn,p(Afc). 

Proof of Lemma 16.31 Proof uses monotonicity of 
the events A^ in the random models G„^p and Gn,m (see 
Lemma 1.10 in [17J. In particular for < pi < p2 < 1 



and < Ml < M2 < N the following inequalities hold: 

On the other hand for any < g < 1 the random graph 
model G(rt, q) conditioning on graphs to have exactly m 
edges is equivalent to the random graph model Grn,m- 
Thus for a random graph G the following holds: 

(1.10) Pn,,(Afc) =P„,q(|S(G)| >m)+¥„,g{\E{G)\ < m) 

N 

< Pn,g(Afe| |£(G)| = £)¥„,,{\E{G)\ = £) 



+ P„,J \E{G)\ <m 



(a) 



JV 



< ^Pn,,(Afci|S(G)| =m)P„,,(|S(G)| 



+ F„,g\^\E{G)\ < ra ^ 

<Pn,m(^fe)+Pn,,(|£(G)| < m) . 

Similarly, one can obtain the following: 
(1.11) P„,,(Afc) > P„,,(Afc n {\E(G)\ < m}) 

m 

= ^p„,,(Afc||i;(G)| = £)P„,,(|i;(G)| =^) 



{b} 



> P„,,(Afe||£;(G)| =m)^P„,,(|i;(G)| =^) 

f=0 

— ^ n.m 

{Ak)Pn,q{\E{G)\ < m) 
>Pn,m(Afe)-P„,g(|£^(G)| > m) 

where both (a) and (6) use monotonicity. Now for a 
small constant /3 > let 



m — m 2 



Pi 



and p2 



1±£ 
m + m 2 



Using Hoeffding's inequality the foUowings hold 



(1.12) 



P„,p, ( |S(G)| >m) < e-^ 
lPn,p. ( |S(G)| <m) < e-^ 



This is due to a variation of the Hoeffding's given in 
that for any < 9 < 1 and < ^ < | gives: 

\E{G)\ ~Nq 



5'^ Nq 

> SNq < e ~. 



Now one can see that by taking 6 
obtain: 



,1 + 13 



■2{m-mi+P) 



we 



,pA \E{G)\>m \ < 



Pn,p,(^\E{G)\ > {l + S)Nq^ 



S^Nq _rr£_ 

< e 4 < e 8 



where (c) uses m > (1 + S)Nq. Similarly we can 
prove the second inequality in p.l2p . Thus we can use 
equations (|1.9p , (jl.lip to obtain the following bound for 

Pn.miAk): 

PnM^k) < P„,p,(v4fe) +P„,p, {\E{G)\ < m) 



< exp 



0(1) -^ICb^ 



r=3 



and similarly using (jl.lOp we get the lower bound: 

VnMAk) > Pn,pi(^fe) +P«,p, {\E{G)\ > m) 



> exp 



0(1) -^ICK 



r=3 



Let H{p) = J2r=3 \'^r\p^ then combining the above two 
inequalities: 

(1.13) e°(i)+^(P)"-f^(Pi) - e^(P)-T^ 



< 



exp [-H{p)] 



< 



^o{l)+H{p)-H{p2) _^ ^H{p)-^^ 

Now we use H{p) = 0{n^°')^ m = v}^"" and apply the 
mean value theorem to H{x) to obtain \H{p) — H{pi)\ — 
\p—pi\H'{p) for p between p, pi. Since for H'(p) we have 
H'{p) = 0(n(*=-i)") and \p,-p\ = -2^^ 
can simplify the equation (|1.13p to obtain the following 

(1.14) 

I exp[o(l) + 0{n' 



'^+^»^+°> +(fc-l)a-2Ni _ lP«,m(^fc) 

>^ exp [-H{p)] 



We would like to have n 
(Mi±^ + (fc - l)a - 2 



ka n 



-00 and 



-2 I i^/i, ' —00 which will give 

us P„,™(^fe) = e°(i)--f^(p). These inequalities can be 
achieved if B satisfies < B < ^-('^^-^'>°' ^nd such B 

" l+a ' 1+Q " 

exist since by assumption a < l/(2fc— 1) that guarantees 
the upper bound for (3 is larger than the lower bound. 
Therefore 



^n.m,k \ — I Pn,m(^fc) 

\ m 



N 



exp [0(1) - H{p)] 



N 



This finishes the proof. 



exp 



iV 



